Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `$r$' of a discrete time Markov chain with `$\mu_{\pi}$' as invariant ergodic measure, possibly having support on an unbounded state space. The main contribution of this paper is circumventing tedious probabilistic methods with a study of a composition of the Markov transition operator $P$ followed by a multiplication operator defined by $e^{r}$. It turns out that even if the observable/ reward function is unbounded, but for some for some $q>2$, $\|e^{r}\|_{q \rightarrow 2} \propto \exp\big(\mu_{\pi}(r) +\frac{2q}{q-2}\big) $ and $P$ is hyperbounded with norm control $\|P\|_{2 \rightarrow q }< e^{\frac{1}{2}[\frac{1}{2}-\frac{1}{q}]}$, sharp non-asymptotic concentration bounds follow. \emph{Transport-entropy} inequality ensures the aforementioned upper bound on multiplication operator for all $q>2$. The role of \emph{reversibility} in concentration phenomenon is demystified. These results are particularly useful for the reinforcement learning and controls communities as they allow for concentration inequalities w.r.t standard unbounded obersvables/reward functions where exact knowledge of the system is not available, let alone the reversibility of stationary measure.
translated by 谷歌翻译
We study the concentration phenomenon for discrete-time random dynamical systems with an unbounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely functional analytic framework. We also show that existence of exponential-type Lyapunov function, compared to the purely deterministic setting, not only implies stability but also exponential concentration inequalities for sampling from the stationary distribution, via \emph{transport-entropy inequality} (T-E). These results have significant impact in \emph{reinforcement learning} (RL) and \emph{controls}, leading to exponential concentration inequalities even for unbounded observables, while neither assuming reversibility nor exact knowledge of random dynamical system (assumptions at heart of concentration inequalities in statistical mechanics and Markov diffusion processes).
translated by 谷歌翻译
Recent works have shown that unstructured text (documents) from online sources can serve as useful auxiliary information for zero-shot image classification. However, these methods require access to a high-quality source like Wikipedia and are limited to a single source of information. Large Language Models (LLM) trained on web-scale text show impressive abilities to repurpose their learned knowledge for a multitude of tasks. In this work, we provide a novel perspective on using an LLM to provide text supervision for a zero-shot image classification model. The LLM is provided with a few text descriptions from different annotators as examples. The LLM is conditioned on these examples to generate multiple text descriptions for each class(referred to as views). Our proposed model, I2MVFormer, learns multi-view semantic embeddings for zero-shot image classification with these class views. We show that each text view of a class provides complementary information allowing a model to learn a highly discriminative class embedding. Moreover, we show that I2MVFormer is better at consuming the multi-view text supervision from LLM compared to baseline models. I2MVFormer establishes a new state-of-the-art on three public benchmark datasets for zero-shot image classification with unsupervised semantic embeddings.
translated by 谷歌翻译
In this study, we propose a lung nodule detection scheme which fully incorporates the clinic workflow of radiologists. Particularly, we exploit Bi-Directional Maximum intensity projection (MIP) images of various thicknesses (i.e., 3, 5 and 10mm) along with a 3D patch of CT scan, consisting of 10 adjacent slices to feed into self-distillation-based Multi-Encoders Network (MEDS-Net). The proposed architecture first condenses 3D patch input to three channels by using a dense block which consists of dense units which effectively examine the nodule presence from 2D axial slices. This condensed information, along with the forward and backward MIP images, is fed to three different encoders to learn the most meaningful representation, which is forwarded into the decoded block at various levels. At the decoder block, we employ a self-distillation mechanism by connecting the distillation block, which contains five lung nodule detectors. It helps to expedite the convergence and improves the learning ability of the proposed architecture. Finally, the proposed scheme reduces the false positives by complementing the main detector with auxiliary detectors. The proposed scheme has been rigorously evaluated on 888 scans of LUNA16 dataset and obtained a CPM score of 93.6\%. The results demonstrate that incorporating of bi-direction MIP images enables MEDS-Net to effectively distinguish nodules from surroundings which help to achieve the sensitivity of 91.5% and 92.8% with false positives rate of 0.25 and 0.5 per scan, respectively.
translated by 谷歌翻译
尽管在零射门学习(ZSL)方面取得了巨大进展,但大多数现有方法仍然依赖于人类通知的属性,这些属性很难注释和扩展。一个无监督的替代方法是使用与其语义类名称相关的单词嵌入来表示每个类。但是,从预训练的语言模型中提取的单词嵌入不一定会捕获视觉相似性,从而导致零拍的性能差。在这项工作中,我们认为在线文本文档,例如Wikipedia,包含有关对象类的丰富视觉描述,因此可以用作ZSL的强大无监督的侧面信息。为此,我们提出了I2Dformer,这是一种基于变压器的新型ZSL框架,共同学会通过在共享嵌入空间中对齐两个方式来编码图像和文档。为了从嘈杂的文档中提取歧视性的视觉单词,我们介绍了一个新的跨模式注意模块,该模块可以学习图像补丁和文档单词之间的细粒度相互作用。因此,我们的i2dformer不仅学习了捕获视觉相似性的高度歧视文档的嵌入,而且还获得了将视觉相关单词定位在图像区域中的能力。定量地,我们证明我们的i2形式在三个公共数据集上的零照片和广义零局学习设置下都显着优于先前无监督的语义嵌入。定性地,我们表明我们的方法会导致高度可解释的结果,其中文档单词可以基于图像区域。
translated by 谷歌翻译
在当今智能网络物理系统时代,由于它们在复杂的现实世界应用中的最新性能,深度神经网络(DNN)已无处不在。这些网络的高计算复杂性转化为增加的能源消耗,这是在资源受限系统中部署大型DNN的首要障碍。通过培训后量化实现的定点(FP)实现通常用于减少这些网络的能源消耗。但是,FP中的均匀量化间隔将数据结构的位宽度限制为大值,因为需要以足够的分辨率来表示大多数数字并避免较高的量化误差。在本文中,我们利用了关键见解,即(在大多数情况下)DNN的权重和激活主要集中在零接近零,只有少数几个具有较大的幅度。我们提出了Conlocnn,该框架是通过利用来实现节能低精度深度卷积神经网络推断的框架:(1)重量的不均匀量化,以简化复杂的乘法操作的简化; (2)激活值之间的相关性,可以在低成本的情况下以低成本进行部分补偿,而无需任何运行时开销。为了显着从不均匀的量化中受益,我们还提出了一种新颖的数据表示格式,编码低精度二进制签名数字,以压缩重量的位宽度,同时确保直接使用编码的权重来使用新颖的多重和处理 - 积累(MAC)单元设计。
translated by 谷歌翻译
数据驱动的方法来协助手术室(OR)工作流程分析取决于耗时且收集昂贵的大型策划数据集。另一方面,我们看到最近从监督学习转变为可以从未标记数据集中学习表示的自我监督和/或无监督学习方法。在本文中,我们利用机器人手术中捕获的未标记数据,并提出了一种新颖的方法,以融合单个视频框架或图像的多模式数据。我们将多模式数据视为不同的观点,而不是同一图像或视频框架的不同图像或视频框架的不同增强(或“视图”)作为不同的观点,可以通过聚类以无监督的方式训练模型。我们将我们的方法与其他最新方法进行了比较,结果表明,我们的方法在手术视频活动识别和语义细分方面的表现出色。
translated by 谷歌翻译
手术视频中的活动识别是开发下一代设备和工作流程监测系统的关键研究领域。由于手术是具有高度变化长度的较长过程,因此用于手术视频的深度学习模型通常包括使用主链和时间序列模型的两阶段设置。在本文中,我们研究了许多最新的骨干和时间模型,以找到为手术活动识别提供最强性能的体系结构。我们首先在大规模活动识别数据集上进行模型性能,该数据集包含在多个临床手术室中捕获的800多个手术视频。我们进一步评估了两个较小的公共数据集(Cholec80和Cataract-101数据集)上的模型,分别包含80个视频和101个视频。我们从经验上发现,Swin-Transformer+BigRU时间模型在两个数据集上都产生了强劲的性能。最后,我们通过对新医院进行微调模型来研究模型对新领域的适应性,并试验最近无监督的域适应方法。
translated by 谷歌翻译
零件代表不同对象的几何和语义相似性的基本单位。我们争辩说,部分知识应与观察到的对象课程中有款组合。对此,我们将3D组成零射击学习作为从看作识的零件泛化的问题,从而看成了语义分割。我们通过将任务与所提出的组成部分数据集进行基准测试,提供结构化研究。该数据集是通过处理原始PartNet来创建的,以最大化不同对象的部分重叠。现有点云部分段方法未能在此设置中概括到未遵守的对象类。作为解决方案,我们提出了分解共识,其将零件分割网络与部分评分网络相结合。我们方法的关键直觉是某些部件的分割掩码应该具有与其部分分数分开的零件分数的共识。在生成最合适的分割掩模之前在每个对象部分中定义的不同部分组合的两个网络原因。我们展示了我们的方法允许组成零射分段和广义零拍分类,并在两个任务中建立最先进的状态。
translated by 谷歌翻译
Diabetic Retinopathy (DR) is considered one of the primary concerns due to its effect on vision loss among most people with diabetes globally. The severity of DR is mostly comprehended manually by ophthalmologists from fundus photography-based retina images. This paper deals with an automated understanding of the severity stages of DR. In the literature, researchers have focused on this automation using traditional machine learning-based algorithms and convolutional architectures. However, the past works hardly focused on essential parts of the retinal image to improve the model performance. In this paper, we adopt transformer-based learning models to capture the crucial features of retinal images to understand DR severity better. We work with ensembling image transformers, where we adopt four models, namely ViT (Vision Transformer), BEiT (Bidirectional Encoder representation for image Transformer), CaiT (Class-Attention in Image Transformers), and DeiT (Data efficient image Transformers), to infer the degree of DR severity from fundus photographs. For experiments, we used the publicly available APTOS-2019 blindness detection dataset, where the performances of the transformer-based models were quite encouraging.
translated by 谷歌翻译